AuditGuard: a system for database auditing under retention restrictions

Auditing the changes to a database is critical for identifying malicious behavior, maintaining data quality, and improving system performance. But an accurate audit log is a historical record of the past that can also pose a serious threat to privacy. In many domains, retention policies govern how long data can be preserved by an institution. Regulations like FERPA and HIPAA (in the U.S.) or the Directive of Data Protection (in the EU), require strict retention periods to be observed, mandating the disposal of past data. In addition, institutions often adopt their own retention policies, choosing to remove sensitive data after a period of time to avoid its unintended release, or to avoid disclosure that could be forced by subpeona. Policies that limit data retention conflict with the goal of accurate auditing, and data owners have to carefully balance the need for accurate auditing with the privacy goals of retention policies. Unfortunately, existing technologies make balancing these goals difficult. Most database systems include audit logs, but there are few mechanisms for limiting access to logs beyond wholesale destruction of the log for a given time period. Some proposed database systems support persistence, in which past states are retained. These can be used for some (but not all) audit functions, and also lack effective protection mechanisms for past versions of data. The AuditGuard system is a flexible database audit mechanism which retains history and enforces retention policy. It allows an auditor to selectively remove data from a log of the past, while still retaining non-sensitive aspects of history that can be vital to an audit. The main components of the AuditGuard system are the following: